Image pickup apparatus and image pickup method used for the same

ABSTRACT

An aspect of the present invention provides an image pickup apparatus, comprising: an image pickup optical system which forms an image of a subject; an imaging device which obtains an image by picking up the image of the subject via the image pickup optical system; a target detection device which detects a size of a target in the image; a first control device which controls a focus position of the image pickup optical system according to the size of the target detected by the target detection device; and a second control device which performs continuous AF that detects a focusing position where a contrast in the image reaches a local maximum by moving the focus position of the image pickup optical system, moves the focus position of the image pickup optical system to the focusing position and keeps an in-focus state.

The present invention is a Continuation of U.S. application Ser. No.12/323,076, which was filed on Nov. 25, 2008, which claims priority toJapanese Application No. 2007-307728, filed on Nov. 28, 2007, the entiredisclosures of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup apparatus that has acontinuous focusing function for performing continuous focusing actions,and an image pickup method used for the same.

2. Description of the Related Art

Various types of cameras having an auto-focus function have been known.

Japanese Patent Application Laid-Open No. 08-248510 discloses a hybridcamera having a still camera part, a video camera part, and a takinglens shared by the parts, wherein, in a still image taking mode, thecamera performs servo control based on repetitive detection by a focusdetection device when a motion detection device detects that a subjectis moving and performs one-shot control that stops a lens drive to makefocus lock in response to attainment of an in-focus state when themotion detection device detects that a subject is not moving.

Japanese Patent Application Laid-Open No. 2003-207712 discloses animaging device that adjusts the focus of optical equipment, including:an optical part that generates a subject image by receiving a light beamfrom a subject; a contrast detection part that detects the contrastvalue of the subject image; a range finding part that computes distanceinformation on the distance from a focusing unit to the subject; anadjustment indication reception part that receives indicationinformation showing the adjustment of the focus of the optical part; afocus control part that adjusts the focus of the optical part when theadjustment indication reception part receives the indicationinformation; and a distance information storage part that stores thecontrast value that the contrast detection part detects and the distanceinformation that the range finding part computes, when the adjustmentindication reception part receives the indication information.

Japanese Patent No. 3934954 (Japanese Patent Application Laid-Open No.2003-232984) discloses a CPU that, before performing a focusing actionin response to focusing instruction operation, compares present imagedata obtained immediately after the focusing instruction operation andobtained when a focus lens is at a previous focusing position detectedby a previous focusing action with a previous image data obtained in themidst of the previous focusing action or immediately after finishing theprevious focusing action and obtained when the focus lens is at theprevious focusing position, and judges whether or not the in-focus statefrom the previous focusing action is maintained. In the case ofdetecting the focusing position when the in-focus state is notmaintained, the focus lens is not driven or the driving area of thefocus lens is narrowed.

SUMMARY OF THE INVENTION

A general auto-focus (AF) function makes focus lock in response toattainment of an in-focus state. That is highly possible to cause anout-of-focus state (so called “pinbokeh (blurry photo)”) when a subjectis moving. A continuous AF function serially performs a focusing actionto achieve highly accurate focusing. Unfortunately, the continuous AFfunction has to find a peak position (the focus lens position where theimage shows the peak contrast) while performing step driving of thefocus lens, which takes much time in focusing.

The present invention intends to ensure both a focus tracking abilityand focusing accuracy for a moving subject.

A first aspect of the present invention provides an image pickupapparatus comprising: an image pickup optical system which forms animage of a subject; an imaging device which obtains an image by pickingup the image of the subject via the image pickup optical system; atarget detection device which detects a size of a target in the image; afirst control device which controls a focus position of the image pickupoptical system according to the size of the target detected by thetarget detection device; and a second control device which performscontinuous AF that detects a focusing position where a contrast in theimage reaches the local maximum by moving the focus position of theimage pickup optical system, moves the focus position of the imagepickup optical system to the focusing position and keeps an in-focusstate.

According to the first aspect, the image pickup apparatus can controlthe focus position of the image pickup optical system according to thesize of the target in the image and also can detect a focusing positionwhere a contrast in the image reaches the local maximum by moving thefocus position of the image pickup optical system and control the focusposition of the image pickup optical system. That enables less number ofsteps in driving the focus lens, ensuring both a focus tracking abilityand focusing accuracy for a moving subject.

A second aspect of the present invention provides the image pickupapparatus according to the first aspect, further comprising: a predictedfocusing position calculation device which calculates a predictedfocusing position onto which light is predicted to be focused based onthe size of the target.

According to the second aspect, the image pickup apparatus can limit arange of step driving the focus lens based on the predicted focusingposition by calculating the predicted focusing position based on thesize of the target. That enables much less number of steps in drivingthe focus lens, enhancing the focus tracking ability.

A third aspect of the present invention provides the image pickupapparatus according to the second aspect, wherein the first controldevice moves the focus position of the image pickup optical system tothe predicted focusing position; and the second control devicedetermines the focusing position based on the contrast in the image bymoving the focus position of the image pickup optical system away fromthe predicted focusing position.

According to the third aspect, the image pickup apparatus detects thefocusing position starting from the predicted focusing position as theorigin. That can ensure both a focus speed and focusing accuracy.

A fourth aspect of the present invention provides the image pickupapparatus according to the second aspect, wherein the first controldevice moves the focus position of the image pickup optical system to aninitial target position near the predicted focusing position; and thesecond control device determines the focusing position based on thecontrast in the image by moving the focus position of the image pickupoptical system from the initial target position toward the predictedfocusing position.

According to the fourth aspect, the image pickup apparatus detects thefocusing position starting around the predicted focusing position as theorigin toward the predicted focusing position. That enables less numberof times to reverse the direction to drive the focus lens for detectingthe focusing position as a whole, enhancing the focus tracking ability.

A fifth aspect of the present invention provides the image pickupapparatus according to any one of the first to the fourth aspects,further comprising: an evaluation value calculating device whichcalculates an evaluation value indicative of the contrast in the image;wherein the first control device does not perform control on the focusposition according to the size of the target if the evaluation valuekeeps less than a referential value.

According to the fifth aspect, the image pickup apparatus prevents wrongfocusing even when a low contrast is kept.

A sixth aspect of the present invention provides the image pickupapparatus according to any one of the first to the fifth aspects,further comprising: a motion detection device which detects a motion ofa subject; wherein the second control device does not perform detectionof the focusing position based on the contrast in the image if thesubject moves faster than a referential speed.

According to the sixth aspect, the image pickup apparatus can balancethe focusing accuracy with the focusing speed (focus tracking ability).

A seventh aspect of the present invention provides the image pickupapparatus according to any one of the first to the sixth aspects,further comprising: a selection device which accepts a selection inputas to which of focusing accuracy and focusing speed is prioritized,wherein the second control device performs the detection of the focusingposition based on the contrast in the image when the focusing accuracyis prioritized, and does not perform the detection of the focusingposition based on the contrast in the image when the focusing speed isprioritized.

According to the seventh aspect, the image pickup apparatus can performfocusing control according to the user's intention.

An eighth aspect of the present invention provides the image pickupapparatus according to any one of the second to the fourth aspects,wherein the first control device and the second control device do notmove the focus position of the image pickup optical system when anabsolute value of a difference between the focus position of the imagepickup optical system and the predicted focusing position is equal to orless than a threshold.

According to the eighth aspect, the image pickup apparatus prevents thefocus lens from being unstably driven (shaken or the like).

A ninth aspect of the present invention provides the image pickupapparatus according to the eighth aspect, wherein the first controldevice moves the focus position of the image pickup optical system to ornear to the predicted focusing position when the absolute value of thedifference between the focus position of the image pickup optical systemand the predicted focusing position is larger than the threshold.

According to the ninth aspect, the image pickup apparatus enhances thefocus tracking ability.

A tenth aspect of the present invention provides the image pickupapparatus according to the eighth or the ninth aspect, furthercomprising: an iris which adjusts the amount of light input into theimaging device via the image pickup optical system, wherein thethreshold is changed according to an iris value of the iris.

According to the tenth aspect, the image pickup apparatus ensures thefocusing accuracy.

An eleventh aspect of the present invention provides the image pickupapparatus according to the eighth or the ninth aspect, wherein the imagepickup optical system has a zoom function, and the threshold is changedaccording to a zoom position or a focal length of the image pickupoptical system.

According to the eleventh aspect, the image pickup apparatus ensures thefocusing accuracy.

A twelfth aspect of the present invention provides the image pickupapparatus according to the eighth or the ninth aspect, wherein thethreshold is changed according to the detected focusing position of theimage pickup optical system or a subject distance corresponding to thefocusing position.

According to the twelfth aspect, the image pickup apparatus ensures thefocusing accuracy.

A thirteenth aspect of the present invention provides the image pickupapparatus according to the second aspect, wherein the predicted focusingposition calculation device calculates the predicted focusing positionbased on a relationship between the size of the target detected by thetarget detection device and the subject distance corresponding to thefocusing position or the focusing position.

According to the thirteenth aspect, the image pickup apparatus cancalculate the predicted focusing position with accuracy, enhancing thefocus tracking ability.

A fourteenth aspect of the present invention provides the image pickupapparatus according to the second aspect, wherein the predicted focusingposition calculation device calculates the predicted focusing positionbased on a ratio of a past size of the target to a present size of thetarget detected by the target detection device.

According to the fourteenth aspect, the image pickup apparatus cancalculate the predicted focusing position with accuracy, enhancing thefocus tracking ability.

A fifteenth aspect of the present invention provides an image pickupmethod used in an image pickup apparatus including: an image pickupoptical system which forms an image of a subject; an imaging devicewhich obtains an image by picking up the image of the subject via theimage pickup optical system; and a target detection device which detectsa size of a target in the image, the image pickup method for performing:a first control of controlling a focus position of the image pickupoptical system according to the size of the target detected by thetarget detection device; and a second control of performing continuousAF that detects a focusing position where a contrast in the imagereaches the local maximum by moving the focus position of the imagepickup optical system, moves the focus position of the image pickupoptical system to the focusing position and keeps an in-focus state.

According to the present invention, the image pickup apparatus canensure both the focus tracking ability and focusing accuracy for amoving subject.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram showing a configuration of a camera;

FIG. 2 is an illustration showing an exemplary image;

FIGS. 3A, 3B, and 3C are illustrations used for explaining facedirections;

FIGS. 4A, 4B, 4C, and 4D are illustrations used for explaining faceangles;

FIG. 5 is a flowchart showing exemplary focusing processing according toa first embodiment;

FIG. 6 is a diagram showing an exemplary focusing position in the firstembodiment;

FIG. 7 is a diagram showing the focusing position determined byinterpolation in the first embodiment;

FIG. 8 is a flowchart showing exemplary focusing processing according toa second embodiment;

FIG. 9 is a diagram showing an exemplary focusing position in the secondembodiment;

FIG. 10 is a diagram showing the focusing position determined byinterpolation in the second embodiment;

FIG. 11 is a first flowchart showing exemplary focusing processingaccording to a third embodiment;

FIG. 12 is a second flowchart showing exemplary focusing processingaccording to the third embodiment;

FIG. 13 is a flowchart showing exemplary focusing processing accordingto a fourth embodiment;

FIG. 14 is a flowchart showing exemplary focusing processing accordingto a fifth embodiment;

FIG. 15 is a flowchart showing exemplary focusing processing accordingto a sixth embodiment;

FIG. 16 is a flowchart showing exemplary focusing processing accordingto a seventh embodiment;

FIG. 17 is a flowchart showing exemplary focusing processing accordingto an eighth embodiment;

FIG. 18 is a flowchart showing exemplary predicted focusing positioncalculation processing according to a ninth embodiment;

FIG. 19 is a diagram exemplifying relationships between pre-focuspositions and the predicted focusing positions;

FIG. 20 is a flowchart showing exemplary predicted focusing positioncalculation processing according to a tenth embodiment; and

FIG. 21 is a diagram showing exemplary drive pulses corresponding to thepresent position and the predicted focusing position of the focus lens.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a schematic block diagram showing a configuration of a camera1 (digital camera) as an example of an image pickup apparatus accordingto the present invention. The camera 1 converts image data obtained byphotography into an image file of Exif format, and records the file in arecording unit 70 such as an external recording medium that can beattached to and detached from the camera.

The digital camera 1 has an operational system that includes: anoperation unit 11 having an operation mode switch, a menu/OK button, azoom/up-down lever, a right-left button, a Back (return) button, adisplay switching button, a release button, a power switch and the like;and an operational system control unit 74 as an interface for sendingthe operation performed on the operation unit 11 to a CPU (CentralProcessing Unit) 75.

A lens 20 includes a focus lens and a zoom lens. The lens 20 can be slidalong an optical axis by a lens drive unit 51. The lens drive unit 51controls movement of the focus lens or the zoom lens based on focusdrive amount data or operated amount data of the zoom/up-down lever onthe operation unit 11, both of which are output from the CPU 75.

An iris 54 is driven by an iris drive unit 55 including a motor and amotor driver. The iris drive unit 55 adjusts a diameter of the irisbased on iris-value data output from an AE/AWB processing unit 63.

An imaging device 58 such as a CCD (Charge Coupled Device) is located atthe rear of an image pickup optical system that includes the lens 20 andthe iris 54. The imaging device 58 has a photoelectric surface on whicha plurality of light receiving devices are arranged two-dimensionally. Alight from the subject passing through the optical system forms an imageon the photoelectric surface and is subjected to photoelectricconversion. Located in front of the photoelectric surface are, amicro-lens array for focusing the light on each pixel and a color filterarray on which filters for R, G, and B colors are regularly arranged.The imaging device 58 outputs an electric charge stored at each of thepixels as a serial analog image signal for each line in synchronizationwith a vertical transfer clock and a horizontal transfer clock suppliedfrom an imaging device control unit 59. A time required by a process ofstoring the electric charge at each of the pixels, that is, an exposuretime is determined by an electronic shutter drive signal provided by theimaging device control unit 59.

The analog image signal output from the imaging device 58 is input to ananalog signal processing unit 60. The analog signal processing unit 60includes a correlated double sampling (CDS) circuit for removing noisefrom the image signal, and an automatic gain controller (AGC) foradjusting a gain of the image signal.

An A/D converter 61 converts the analog image signal that is processedthrough the analog signal processing unit 60 into a digital image signal(hereinafter, referred to as “image data”). The image data is CCD-RAWdata having density values of R, G, and B colors for each of the pixels.

A control circuit 72 generates a timing signal, feeds the timing signalto the imaging device control unit 59, and synchronizes operation on therelease button in the operation unit 11, input of the electric charge ofthe imaging device 58 and processing by the analog signal processingunit 60.

A flash control unit 73 causes a flash 24 to emit the light at the timeof photography. Specifically, when the flash emission mode is set toflash-on, the flash control unit 73 turns on the flash 24 to emit thelight at the time of photography. When the flash emission mode is set toflash-off, the flash control unit 73 disables the flash 24 from emittingthe light at the time of photography.

The image data output from the A/D converter 61 is subjected towhite-balance (WB) adjustment, gamma correction, and YC processing at adigital signal processing unit 65, and written to a memory 66. TheCCD-RAW data is also written to the memory 66.

The memory 66 is a memory used as workspace for various kinds of digitalimage processing on the image data. The memory 66 may be an SDRAM(Synchronous Dynamic Random Access Memory) that carries out datatransfer in synchronization with a bus clock signal of a predeterminedperiod, for example.

A display unit 71 is used for displaying the image data that is seriallystored in the memory 66 since the photography mode is set until thephotography instruction is issued as through images, and for displayingthe image data saved in the recording unit 70 in the playback mode. Thethrough image is photographed by the imaging device 58 at predeterminedintervals when the photography mode is on.

An AF processing unit 62 and the AE/AWB processing unit 63 determine aphotography condition based on the image data.

The AF processing unit 62 detects the focusing position based on theimage data and outputs the focus drive amount data (AF processing). Fordetecting the focusing position, a contrast detection method is adopted.The method takes advantage of such a nature of the image as havinghigher contrast in the accurate focus in detecting the focusingposition. The AF processing unit 62 obtains an AF evaluation value byextracting high frequency components from the image data and summingthem for the entire image or a part of the entire image (such as acenter part or a face detecting region). The AF processing unit 62detects the focus lens position where the AF evaluation value thatindicates the contrast reaches the local maximum by moving the focuslens position in the lens 20 (also referred to as “focus position”) withthe lens drive unit 51. The AF processing unit 62 determines thedetected focus lens position as the focusing position. In the presentinvention, the AF evaluation value may also be referred to as “contrastevaluation value” or “focusing evaluation value”.

The AE/AWB processing unit 63 measures the subject luminance based onthe image data, and determines an iris value, a shutter speed, and thelike based on the measured subject luminance. The AE/AWB processing unit63 determines the iris value data and the shutter speed data as exposuresetting values (AE processing). The AE/AWB processing unit 63 alsodetermines the amount of correction performed on the white-balance ofthe image data based on the image data (AWB processing).

A ROM 68 stores various constants set in the camera 1, a program the CPU75 runs and the like. A RAM 69 temporarily stores data needed by the CPU75 in running a program.

The CPU 75 controls each of the units of the camera 1.

A face detection unit 80 detects a human face from the image data.Specifically, the face detection unit 80 detects a region in a facehaving characteristics of face (for example, the skin color, aparticular part of the face such as eyes, and the face shape), as a faceregion, which is not necessarily limited to this method.

The face detection unit 80 also detects the face size in the image data.In an entire image 201 (also referred to as “screen”) shown in FIG. 2,the ratio (Lf/Lv) is calculated as the face size information where Lv isthe number of pixels counted in the vertical direction of the entireimage 201 and Lf is the number of pixels counted lengthwise of the faceregion 202. Specifically, the ratio of a dimension of the face image toa dimension of the entire image 201 is calculated as the face sizeinformation. Although FIG. 2 shows a simple example for facilitating theunderstanding of the present invention, the face may be looking into anydirection such as looking left, looking front, looking right as shown inFIG. 3A to 3C. The face may be tilted at any angle as shown in FIG. 4Ato 4D. Therefore, the face information to be obtained here needs to bethe same for the same distance from the subject, dependent neither onthe direction nor on the tilted angle. It may be preferable to obtainthe length of the face after detecting the direction and the tiltedangle of the face based on physical relationships between the face parts(eyes, mouth and the like). Alternatively, the ratio of a distancebetween the face parts (for example, a distance between the eyes and themouth, a distance between the eyes, etc.) to the dimension in the entireimage may be calculated as the face size information by using thedistance between the face parts in the place of the dimension of theentire face. The direction, the tilted angle, and the size of the facemay be detected by any known methods including that disclosed by theapplicant in the Japanese Patent Application Laid-Open No. 2007-34261.

A motion detection unit 83 detects a motion of the subject based on theimage data. Specifically, the motion detection unit 83 obtains a motionvector in the face area (face region) of the image data. The motionvector indicates the direction and the speed of the motion of the facearea in the image data (the speed relative to the image pickupapparatus).

A predicted focusing position calculation unit 84 calculates a predictedfocusing position where the image is predicted to be in focus for thefocus lens position (focus position) in the lens 20. The specificcalculation will be detailed later.

Although the embodiment has been described by taking an example of thecase where a face is the target to be focused, the target is not limitedto the face and may be other than the face. It is needless to say thatthe camera may be provided with a detection unit for targets other thanthe face and a detection unit for a motion of targets other than theface.

The CPU 75 in the control circuit 72 may function as the AF processingunit 62, the AE/AWB processing unit 63, the face detection unit 80, themotion detection unit 83, and the predicted focusing positioncalculation unit 84.

Various focusing processing will be detailed below with reference torespective embodiments.

First Embodiment

FIG. 5 is a flowchart showing a flow of exemplary focusing processingaccording to the first embodiment in an out-of-focus state. The focusingprocessing is performed by the AF processing unit 62 under generalcontrol of the CPU 75 of the camera 1.

At step S11, the AF processing unit 62 calculates the predicted focusingposition (the focus lens position where the image becomes in focus) ofthe lens 20 by the predicted focusing position calculation unit 84. Inthe embodiment, the target is a face whose predicted focusing positionis calculated from the size of the face area detected by the facedetection unit 80.

At step S12, the AF processing unit 62 slides the focus lens to thepredicted focusing position by the lens drive unit 51. At step S13, theAF processing unit 62 searches for the focus lens position where the AFevaluation value that indicates the contrast in the image reaches thelocal maximum (hereinafter, referred to as “peak position”) by movingthe focus lens position away from the predicted focusing position withthe lens drive unit 51.

At step S14, the AF processing unit 62 judges whether or not the peakposition is found. If the peak position is found, the operation proceedsto step S15. If the peak position is not found, the operation proceedsto step S18.

At step S15, the AF processing unit 62 determines the peak position asthe focusing position.

At step S16, the AF processing unit 62 slides the focus lens to thefocusing position (i.e., the peak position) by the lens drive unit 51.

At step S17, the AF processing unit 62 sets a focusing state flag on.Hereinafter, the state where the focusing state flag is set on isreferred to as “in-focus state”.

At step S18, the AF processing unit 62 keeps the out-of-focus state. Theout-of-focus state refers to a state where the focusing state flag isset off.

FIG. 6 exemplifies a relationship between the focus lens positions andthe AF evaluation values and how the focus lens is driven (focusdriving) in the embodiment. In the embodiment, the AF processing unit 62detects the focusing position based on the AF evaluation values bymoving the focus lens position away from the predicted focusingposition.

First, the AF processing unit 62 calculates the predicted focusingposition, and then slides the focus lens to the predicted focusingposition. Next, the AF processing unit 62 obtains the AF evaluationvalues near the predicted focusing position and finds the focus lensposition where the AF evaluation value reaches the peak (peak position).The AF processing unit 62 determines the found peak position as thefocusing position and slides the focus lens to the focusing position.

FIG. 7 exemplifies a relationship between the focus lens positions andthe AF evaluation values and how the focus lens is driven (focusdriving) in the case where the focusing position is detected with higheraccuracy by taking finer step width than that taken to slide the focuslens in detecting the peak position. With interpolation to obtain the AFevaluation values by using the AF evaluation values at adjoining focuspositions and the focus positions, the focusing position is calculated.

Second Embodiment

FIG. 8 is a flowchart showing a flow of exemplary focusing processingaccording to the second embodiment in an out-of-focus state. Theprocessing is performed by the AF processing unit 62 under generalcontrol of the CPU 75 of the camera 1.

Step S21 is the same as step S11 of the first embodiment.

At step S22, the AF processing unit 62 slides the focus lens near to thepredicted focusing position (hereinafter, referred to as “initial targetposition”) by the lens drive unit 51.

At step S23, the AF processing unit 62 searches for the focus lensposition where the AF evaluation value that indicates the contrast inthe image reaches the local maximum (peak position) by moving the focuslens position from the initial target position toward the predictedfocusing position with the lens drive unit 51.

Steps S24 to S28 are the same as steps S14 to S18 of the firstembodiment.

FIG. 9 exemplifies a relationship between the focus lens positions andthe AF evaluation values and how the focus lens is driven (focusdriving) in the embodiment. In the embodiment, the AF processing unit 62detects the focusing position based on the AF evaluation values bymoving the focus lens position from the initial target position near thepredicted focusing position toward the predicted focusing position.

First, the AF processing unit 62 calculates the predicted focusingposition, sets the position near the predicted focusing position as theinitial target position, and slides the focus lens to the initial targetposition. Next, the AF processing unit 62 obtains the AF evaluationvalues near the predicted focusing position and finds the focus lensposition where the AF evaluation value reaches the peak (peak position).The AF processing unit 62 determines the found peak position as thefocusing position and slides the focus lens to the focusing position.

In some cases, the number of times to reverse the direction of slidingthe focus lens may be less than the first embodiment shown in FIG. 6.That means that the focus tracking ability may be better than that ofthe first embodiment.

FIG. 10 exemplifies a relationship between the focus lens positions andthe AF evaluation values and how the focus lens is driven (focusdriving) in the case where the focusing position is detected with higheraccuracy by taking finer step width than that taken to slide the focuslens in detecting the peak position. With interpolation to obtain the AFevaluation values by using the AF evaluation values at adjoining focuspositions and the focus positions, the focusing position is calculated.

Third Embodiment

FIG. 11 is a flowchart showing a flow of exemplary focusing processingaccording to the third embodiment in an out-of-focus state. Theprocessing is performed by the AF processing unit 62 under generalcontrol of the CPU 75 of the camera 1. In FIG. 11, the steps same asthose in the first embodiment shown in FIG. 5 are designated by the samereference symbols and the detailed description of the steps is omitted.

At step S10, the AF processing unit 62 judges whether a focusingprediction flag is off or not. The focusing prediction flag is set on asthe default. The case where the focusing prediction flag is set off willbe described later. If the focusing prediction flag is set on, theoperation proceeds to step S11. If the focusing prediction flag is setoff, the operation proceeds to step S31.

Steps S11 to S14 are the same as those in the first embodiment.

If the peak position is found, the operation proceeds to step S15. Ifthe peak position is not found, the operation proceeds to step S35.

Steps S15 to S17 are the same as those in the first embodiment.

If the focusing prediction flag is off, the AF processing unit 62neither calculates the predicted focusing position (step S11) nor slidesthe focus lens to the predicted focusing position (step S12). That meansthat the focus lens 10 is not slid in accordance with the face size inthe image when the focusing prediction flag is off.

If the focusing prediction flag is off, the AF processing unit 62 doesnot slide the focus lens to the predicted focusing position, and at stepS31, searches for the peak position based on the AF evaluation values.Specifically, the AF processing unit 62 searches for the peak positionwhere the AF evaluation value reaches the local maximum by moving thefocus lens position away from the present position with the lens driveunit 51.

At step S32, the AF processing unit 62 judges whether the peak positionis found or not. If the peak position is found, the operation proceedsto step S33. If the peak position is not found, the operation proceedsto step S34.

At step S33, the AF processing unit 62 determines the peak position asthe focusing position, and then the operation proceeds to step S16.

At step S34, the AF processing unit 62 sets the focusing prediction flagon. At step S35, the AF processing unit 62 keeps the out-of-focus state.

FIG. 12 is a flowchart showing a flow of exemplary focusing processingaccording to the third embodiment in an in-focus state. The processingis performed by the AF processing unit 62 under general control of theCPU 75 of the camera 1.

At step S301, the AF processing unit 62 judges whether a state of a lowAF evaluation value has continued in time series or not. Specifically,the AF processing unit 62 judges whether a state of the image with lowcontrast has continued for the number of image pickup times (or acertain period of time, or the number of determinations), indicated by athreshold, or more.

If the state of the low AF evaluation value has continued, the operationproceeds to step S312. If the state of the low AF evaluation value hasnot continued, the operation proceeds to step S302.

At step S302, the AF processing unit 62 calculates the predictedfocusing position based on the size of the target in the image (forexample, a face) with the predicted focusing position calculation unit84.

At step S304, the AF processing unit 62 judges whether a difference (anabsolute value) between the present focus lens position and thepredicted focusing position is larger than a motion judgment thresholdor not. In the present invention, the focus lens position may also bereferred to as “FOCUS position” or “focus position”. For the motionjudgment threshold, a value corresponding to the depth of field is used.

If the difference is larger than the motion judgment threshold, it isdetermined that the present focus lens position cannot keep the in-focusstate. Specifically, the AF processing unit 62 determines that the focuslens needs to be slid to a suitable position if the difference isoutside the range of the depth-of-field correspondent values.

If the difference is judged as larger than the motion judgment thresholdat the step S304, the operation proceeds to step S307. If the differenceis judged as equal to or less than the motion judgment threshold at thestep S304, the operation proceeds to step S305.

At step S307, the AF processing unit 62 slides the focus lens to thepredicted focusing position by the lens drive unit 51.

At step S308, the AF processing unit 62 searches for the peak positionwhere the AF evaluation value reaches the local maximum by sliding thefocus lens.

At step S309, the AF processing unit 62 judges whether the peak positionis found or not. If the peak position is found, the operation proceedsto step S310. If the peak position is not found, the operation proceedsto step S313.

At step S310, the AF processing unit 62 determines the peak position asthe focusing position. At step S311, the AF processing unit 62 slidesthe focus lens to the focusing position (i.e., the peak position).

If the difference is judged as equal to or less than the motion judgmentthreshold at the step S304, the AF processing unit 62 does not slide thefocus lens (step S305) and keeps the in-focus state (step S306).Specifically, if the AF processing unit 62 judges that the differencebetween the present focus position and the predicted focusing positionis within the range of the depth-of-field correspondent values, the AFprocessing unit 62 does not slide the focus lens. That can prevent thefocus lens from being unstably driven (shaken or the like) in thein-focus state.

If the AF processing unit 62 judges that the state of the low AFevaluation value has continued at step S301, the AF processing unit 62sets the focusing prediction flag off at step S312 and enters into theout-of-focus state at step S313. In the out-of-focus state, i.e., if thefocusing prediction flag is off as described with reference to FIG. 11,the AF processing unit 62 does not calculate the predicted focusingposition (step S11 in FIG. 11) and determines the focusing positionbased on the AF evaluation values (step S31 in FIG. 11).

Here, after the AF processing unit 62 detects the focusing position byusing the predicted focusing position in the out-of-focus state, thestate of the low AF evaluation value (the state of the low contrast) maycontinue. That may occur in such a case where the AF processing unit 62predicts that a position largely different from the correct focusingposition will be the focusing position. It is assumed that an image of aface printed on a business card is picked up. As the face printed on thebusiness card is much smaller than the actual face, the AF processingunit 62 may calculate the position quite different from the correctfocusing position as the predicted focusing position. In addition, ifthe AF processing unit 62 finds a peak caused by a noise or the likenear such a wrong predicted focusing position, the AF processing unit 62slides the focus lens to a position largely different from the correctfocusing position. In order to cope with the situation, the embodimentis adapted to disable the focusing position prediction, only perform thefocusing position detection based on the AF evaluation value and slidethe focus lens to the detected focusing position, if the state of thelow AF evaluation value continues.

Fourth Embodiment

FIG. 13 is a flowchart showing a flow of focusing processing accordingto the fourth embodiment in an in-focus state. The processing isperformed by the AF processing unit 62 under general control of the CPU75 of the camera 1. In FIG. 13, the steps same as those in the thirdembodiment shown in FIG. 12 are designated by the same reference symbolsand the detailed description of the steps is omitted.

Steps S301 to S302 and steps S304 to S312 are the same as those in thethird embodiment.

In the in-focus state, the AF processing unit 62 keeps obtaining sizeinformation on the target (for example, the face) in the image andcalculating the predicted focusing position. While performing theprocessing, the AF processing unit 62 obtains the threshold for judgingwhether the focus lens position is to be moved or not (motion judgmentthreshold) at step S303 a.

In the embodiment, a LUT (lookup table) showing a relationship betweeniris F values and motion judgment thresholds is stored in the controlcircuit 72 in advance. The motion judgment threshold corresponding tothe present iris F value (iris value) is obtained from the LUT. If thedifference between the present focus lens position and the predictedfocusing position is equal to or less than the motion judgmentthreshold, the focus lens is not driven (step S305). That means if thedifference is judged as within the range of the depth of field, thefocus lens is not driven.

The depth of field varies according to the iris F value. In the in-focusstate, the AF processing unit 62 can perform accurate focus control bychanging the motion judgment threshold for judging whether the focuslens position is to be moved or not according to the present iris Fvalue.

Fifth Embodiment

FIG. 14 is a flowchart showing a flow of focusing processing accordingto the fifth embodiment in an in-focus state. The processing isperformed by the AF processing unit 62 under general control of the CPU75 of the camera 1. In FIG. 14, the steps same as those in the thirdembodiment shown in FIG. 12 are designated by the same reference symbolsand the detailed description of the steps is omitted.

Steps S301 to S302 and steps S304 to S313 are the same as those in thethird embodiment.

In the in-focus state, the AF processing unit 62 keeps obtaining sizeinformation on the target (for example, the face) in the image andcalculating the predicted focusing position. While performing theprocessing, the AF processing unit 62 obtains the threshold for judgingwhether the focus lens position is to be moved or not (motion judgmentthreshold) at step S303 b.

In the embodiment, a LUT (lookup table) showing a relationship betweenthe zoom lens positions corresponding to the focal lengths and motionjudgment thresholds is stored in the control circuit 72 in advance. Themotion judgment threshold corresponding to the present zoom position isobtained from the LUT. In the present invention, the zoom lens positionmay be referred to as “zoom position”. If the difference between thepresent focus lens position and the predicted focusing position is equalto or less than the motion judgment threshold, the focus lens is notdriven (step S305). That means if the difference is judged as within therange of the depth of field, the focus lens is not driven.

The depth of field varies according to the zoom lens position (or thefocal length). In the in-focus state, the AF processing unit 62 canperform accurate focus control by changing the motion judgment thresholdfor judging whether the focus lens position is to be moved or notaccording to the present zoom lens position (or the focal length).

Sixth Embodiment

FIG. 15 is a flowchart showing a flow of focusing processing accordingto the sixth embodiment in an in-focus state. The processing isperformed by the AF processing unit 62 under general control of the CPU75 of the camera 1. In FIG. 15, the steps same as those in the thirdembodiment shown in FIG. 12 are designated by the same reference symbolsand the detailed description of the steps is omitted.

Steps S301 to S302 and steps S304 to S313 are the same as those in thethird embodiment.

In the in-focus state, the AF processing unit 62 keeps obtaining sizeinformation on the target (for example, the face) in the image andcalculating the predicted focusing position. While performing theprocessing, the AF processing unit 62 obtains the threshold for judgingwhether the focus lens position is to be moved or not (motion judgmentthreshold) at step S303 c.

In the embodiment, a LUT (lookup table) showing a relationship betweenthe focus lens positions and motion judgment thresholds is stored in thecontrol circuit 72 in advance. The motion judgment thresholdcorresponding to the present focus lens position is obtained from theLUT. In the present invention, the focus lens position may be referredto as “FOCUS position” or “focus position”. If the difference betweenthe present focus lens position and the predicted focusing position isequal to or less than the motion judgment threshold, the focus lens isnot driven (step S305). That means if the difference is judged as withinthe range of the depth of field, the focus lens is not driven.

The depth of field varies according to the focus lens position (or thesubject distance). In the in-focus state, the AF processing unit 62 canperform accurate focus control by changing the motion judgment thresholdfor judging whether the focus lens position is to be moved or notaccording to the present focus lens position (or the subject distance).

Seventh Embodiment

When the subject (target) is moving, detecting the peak position basedon the contrast in the image is inconvenient for the reasons below.

First, the AF processing unit 62 detects the focusing position bysliding the focus lens back and forth along the optical axis.Accordingly, the focus lens always reaches the focusing position alittle bit later than the motion of the subject.

Second, when the subject is moving, the image is sometimes blurred andin a low contrast. For such an image, it cannot be expected to haveaccurate focusing position detection.

In order to cope with the situation, the embodiment is adapted todisable the peak position detection based on the contrast in the imageand serially slide the focus lens to the predicted focusing position, ifthe subject is moving too fast. Accordingly, when the motion of thesubject is too large, the embodiment sometimes cannot set the focus lensto the correct focusing position; though, the embodiment can make thefocus lens follow the motion of the subject with focusing accuracy tosome extent.

FIG. 16 is a flowchart showing a flow of focusing processing accordingto the seventh embodiment in an out-of-focus state. The processing isperformed by the AF processing unit 62 under general control of the CPU75 of the camera 1.

Steps S40 and S41 are the same as steps S11 and S12 of the firstembodiment.

At step S42, the AF processing unit 62 detects the motion vector of thesubject (target) with the motion detection unit 83. By detecting themotion vector of the subject, the AF processing unit 62 can detect thepresence and the speed of the motion of the subject.

At step S43, the AF processing unit 62 judges whether the subject ismoving fast or not. Specifically, the AF processing unit 62 compares thespeed of the motion with the motion judgment threshold. If it is judgedthat the subject is moving fast (i.e., the difference between thepresent focus lens position and the predicted focusing position islarger than the motion judgment threshold), the operation proceeds tostep S44. If it is judged that the subject is moving slow (or thesubject is not moving), the operation proceeds to step S49.

At step S44, the AF processing unit 62 drives the focus lens with thelens drive unit 51 and searches for the peak position by sliding thefocus lens away from the predicted focusing position.

At step S45, the AF processing unit 62 judges whether the peak positionis found or not. When the peak position is found, the operation proceedsto step S46. When the peak position is not found, the operation proceedsto step S49.

At step S46, the AF processing unit 62 determines the peak position asthe focusing position. At step S47, the AF processing unit 62 slides thefocus lens to the focusing position with the lens drive unit 51. At stepS48, the AF processing unit 62 sets the focusing state flag on. At stepS49, the AF processing unit 62 keeps the out-of-focus state.

A nonlimiting example of a method for detecting the motion of thesubject include a method of detecting the motion based on an imagesignal obtained by an image pickup unit. For example, the embodimentdetects a target (for example, the face) as a primary subject, and thenestimates an optical flow by using the correlation method or KLT basedmethods. The embodiment may be adapted to determine a motion from theamount of change in the photometric value or the amount of change in theAF evaluation value instead of installing a particular motion detectionmethod.

Eighth Embodiment

The focusing position of the focus lens can be detected based on thecontrast in the image with high accuracy for a static object but with apoor focus tracking ability for a moving object. In order to cope withthe situation, the embodiment is adapted to accept a selection input bythe user as to which of the focusing speed and the focusing accuracy isprioritized, and based on the selection, switch enabling and disablingto detect the focusing position based on the contrast in the image.

FIG. 17 is a flowchart showing a flow of focusing processing accordingto the eighth embodiment in an out-of-focus state. The processing isperformed by the AF processing unit 62 under general control of the CPU75 of the camera 1.

Steps S51 and S52 are the same as steps S11 and S12 of the firstembodiment.

At step S53, the AF processing unit 62 judges which of the focusingspeed and the focusing accuracy the user prioritizes. When the userinputs the selection between the focusing speed and the focusingaccuracy from the operation unit 11, the user's selection is stored inthe control circuit 72. At step S53, the AF processing unit 62references the user's selection stored in the control circuit 72.

At step S54, the AF processing unit 62 drives the focus lens with thelens drive unit 51, and searches for the peak position by sliding thefocus lens away from the predicted focusing position.

At step S55, the AF processing unit 62 judges whether the peak positionis found or not. If the peak position is found, the operation proceedsto step S56. If the peak position is not found, the operation proceedsto step S59.

At step S56, the AF processing unit 62 determines the peak position asthe focusing position. At step S57, the AF processing unit 62 slides thefocus lens to the focusing position with the lens drive unit 51. At stepS58, the AF processing unit 62 sets the focusing state flag on. At stepS49, the AF processing unit 62 keeps the out-of-focus state.

For the user's selection input, the embodiment may be adapted to enablethe user to select from the focusing speed prioritizing mode and thefocusing accuracy prioritizing mode on a menu screen or the like. Theoptions on the menu screen are not limited for directly asking the userwhich of the focusing speed and the focusing accuracy the userprioritizes, and may be for asking the user whether the subject is amoving object (equal to the case of the focusing speed prioritizingmode) or a static object (equal to the case of the focusing accuracyprioritizing mode).

Ninth Embodiment

The embodiment has a flow of focusing processing roughly the same as anyof those in the first to the eighth embodiments. In the embodiment, atthe step of calculating the predicted focusing position, the predictedfocusing position is calculated based on the change in the pre-focusposition. The “pre-focus position” is the focus lens position estimatedfrom the size of the target in the image. For a face as the target, itis particularly referred to as “face pre-focus position”.

FIG. 18 is a flowchart showing a flow of exemplary predicted focusingposition calculation processing according to the embodiment.

At step S81, the AF processing unit 62 calculates the pre-focus position(Pulse_Target_old) based on the size of the target in the image measuredwhen the image enters the in-focus state. For a face as the target, theface pre-focus position may be obtained by the technique disclosed bythe applicant in Japanese Patent Application Laid-Open No. 2007-34261 inand after the paragraph 0240.

At step S82, the AF processing unit 62 calculates a new pre-focusposition (Pulse_Target) based on the size of the target in the presentimage. For a face as the target, the face pre-focus position may beobtained by the above described technique.

At step S83, the AF processing unit 62 calculates the predicted focusingposition from the predicted focusingposition=Pulse_Target—(Pulse_Target_old—focusing position). The focusingposition is the peak position detected based on the contrast in theimage.

State (a) in FIG. 19 exemplifies the focusing position fa detected whenthe image enters the in-focus state. State (b) in FIG. 19 exemplifiesthe focusing position fa and the face pre-focus position fb(“Pulse_Target old” in FIG. 18) detected when the image enters thein-focus state. State (c) in FIG. 19 exemplifies the new face pre-focusposition fc (“Pulse_Target” in FIG. 18). State (d) in FIG. 19exemplifies the new face pre-focus position fc and the predictedfocusing position fd. Through states (a) to (d) in FIG. 19, the abscissaindicates the focus lens position (focus position) in drive pulses fromthe origin position HP (home position for the focus lens).

The AF processing unit 62 calculates the drive pulse from the originposition HP of the focus lens to a past face pre-focus position Pb basedon the size of the target (the face in the embodiment) in the imagemeasured when the image enters the in-focus state, and obtains anestimated difference for the face pre-focus position (fb−fa) based onthe difference between the drive pulse from HP to Pb and the drive pulsefrom HP to the focusing position Pa (FIGS. 19A, 19B).

Then, the AF processing unit 62 obtains the new face pre-focus positionfc based on a new face detection result ((c) in FIG. 19).

The AF processing unit 62 determines the value of the new face pre-focusposition fc corrected with the estimated difference for the facepre-focus position (fb−fa) as the predicted focusing position(fd=fc−(fb−fa)) ((d) in FIG. 19).

Specifically, the AF processing unit 62 obtains the predicted focusingposition fd (=fc−(fb−fa)) by obtaining the difference (fb−fa) betweenthe focusing position fa (the peak position obtained based on thecontrast in the image as the focus lens position is being moved) and thepre-focus position fb corresponding to the focusing position fa (thefocus lens position calculated based on the size of the target inpresent image measured when the image enters the in-focus state), andcorrecting the new face pre-focus position fc (the focus lens positioncalculated based on the target in the present image) with thatdifference. In other words, the AF processing unit 62 obtains thepredicted focusing position fd (=fa+(fc−fb)) based on the change in thepre-focus position (fc−fb).

Although the embodiment has been described with the face as an exampleof the target, the target needs not to be limited to the face. With arelationship between size information on a particular target in thepicked up image and the distance from the subject or the focusingposition of the focus lens (for example, the drive amount), thepredicted focusing position can also be calculated with a target otherthan the face.

Tenth Embodiment

The embodiment has a flow of focusing processing roughly the same as anyof those in the first to the eighth embodiments. In the embodiment, thepredicted focusing position is calculated based on the change in thesize of the target in the image.

FIG. 20 is a flowchart showing a flow of exemplary predicted focusingposition calculation processing according to the embodiment. In theprocessing:

At step S91, the AF processing unit 62 obtains the size information onthe target in the image measured when the image enters the in-focusstate Pa_old (for example, a ratio of the target to the image). When thepicked up entire image 201 shown in FIG. 2 has the face as the target, aratio (Lf/Lv) of the length Lf of the face region 202 to the length Lvof the entire image 201 is used as the face size information.

At step S92, the AF processing unit 62 obtains the size information onthe target in the present image Pa. For a face as the target, the sizeinformation on the face may be obtained by the above describedtechnique.

At step S93, the AF processing unit 62 calculates the predicted focusingposition by the predicted focusing position=infinity positionINF+na*Pa/Pa_old. The infinity position INF is the drive pulse from theorigin position HP to the infinity position INF. na is a pulse from theinfinity position INF to the focusing position (present position). Thefocusing position is the peak position detected based on the contrast inthe image.

FIG. 21 exemplifies the origin position HP of the focus lens, theinfinity position INF, the present position p1, the predicted focusingposition p2, the drive pulse na from the infinity position INF to thepresent position p1, and the drive pulse nb from the infinity positionINF to the predicted focusing position p2. The present position p1 is afocusing position detected in the past. The abscissa indicates the focuslens position (focus position) in drive pulses from the origin positionHP.

The drive pulse from the infinity position INF to the focusing positionis in proportion to the reciprocal of the subject distance. As the sizeof the target in the image is in inverse proportion to the subjectdistance, the drive pulse from the infinity position INF to the focusingposition is in proportion to the size of the subject. Thus, the drivepulse na from the infinity position INF to p1 multiplied by the rate ofchange in the size of the target in the image (Pa/Pa_old) (na*Pa/Pa_old)is the drive pulse nb from the infinity position INF to the focusingposition p2.

Accordingly, the predicted focusing position p2=INF+na*Pa/Pa_old.

The present invention has been described by taking examples from thefirst embodiment to the ninth embodiment for easier understanding;though, it is needless to say that the present invention is not limitedto implementation of each of the embodiments and may include anycombination of the embodiments or application of the important part ofother embodiments.

It should be understood that the present position is not limited to theexamples described in the specification and shown in the figures, andvarious alterations and improvements may be performed without departingfrom the spirit of the present invention.

1. An image pickup apparatus, comprising: an image pickup optical systemwhich forms an image of a subject; an imaging device which obtains animage by picking up the image of the subject via the image pickupoptical system; a target detection device which detects a size of atarget in the image; a first control device which controls a focusposition of the image pickup optical system according to the size of thetarget detected by the target detection device; and a second controldevice which performs continuous AF that detects a focusing positionwhere a contrast in the image reaches a local maximum by moving thefocus position of the image pickup optical system, moves the focusposition of the image pickup optical system to the focusing position andkeeps an in-focus state.
 2. The image pickup apparatus according toclaim 1, further comprising a predicted focusing position calculationdevice which calculates a predicted focusing position onto which lightis predicted to be focused based on the size of the target.
 3. The imagepickup apparatus according to claim 2, wherein the first control devicemoves the focus position of the image pickup optical system to thepredicted focusing position, and the second control device determinesthe focusing position based on the contrast in the image by moving thefocus position of the image pickup optical system away from thepredicted focusing position.
 4. The image pickup apparatus according toclaim 2, wherein the first control device moves the focus position ofthe image pickup optical system to an initial target position near thepredicted focusing position, and the second control device determinesthe focusing position based on the contrast in the image by moving thefocus position of the image pickup optical system from the initialtarget position toward the predicted focusing position.
 5. The imagepickup apparatus according to claim 1, further comprising: an evaluationvalue calculating device which calculates an evaluation value indicativeof the contrast in the image, wherein the first control device does notperform control on the focus position according to the size of thetarget if the evaluation value keeps less than a referential value. 6.The image pickup apparatus according to claim 3, further comprising: anevaluation value calculating device which calculates an evaluation valueindicative of the contrast in the image, wherein the first controldevice does not perform control on the focus position according to thesize of the target if the evaluation value keeps less than a referentialvalue.
 7. The image pickup apparatus according to claim 4, furthercomprising: an evaluation value calculating device which calculates anevaluation value indicative of the contrast in the image, wherein thefirst control device does not perform control on the focus positionaccording to the size of the target if the evaluation value keeps lessthan a referential value.
 8. The image pickup apparatus according toclaim 2, wherein the predicted focusing position calculation devicecalculates the predicted focusing position based on the focusingposition or a relationship between the size of the target detected bythe target detection device and the subject distance corresponding tothe focusing position.
 9. An image pickup method used in an image pickupapparatus including an image pickup optical system which forms an imageof a subject, an imaging device which obtains an image by picking up theimage of the subject via the image pickup optical system, and a targetdetection device which detects a size of a target in the image, theimage pickup method for performing: a first control of controlling afocus position of the image pickup optical system according to the sizeof the target detected by the target detection device; and a secondcontrol of performing continuous AF that detects a focusing positionwhere a contrast in the image reaches the local maximum by moving thefocus position of the image pickup optical system, moves the focusposition of the image pickup optical system to the focusing position andkeeps an in-focus state.